The cell below contains packages that might need to be installed if not done so already. Uncomment for a one time installation.

Imports.

Chess : Data Analysis & Visualization

Data provided by Lichess API tool


Hello! This notebook is designed to help you gain some insight into your chess play, whether you are a new or seasoned player. Using your LiChess account, you will be able to get many visualizations and tables that provide information and comparisons with other users on win/loss, opening play, clock times and engine evaluations.

That being said, there is quite a bit of work that needs to be done in order to use this notebook, mainly corresponding to the first section of the notebook, Setup. That section will walk you through all of the work you need to do in order to get everything running, but to summarize, you will need to download your games off of https://lichess.org/api and use the replace feature in the case of errors when parsing the file.

The first section of the notebook converts your PGN file into a CSV file (table); once you have the CSV file, you do not need to run the first section until you get new data (or equivalently, a new PGN file).

The second section must be run every time the notebook is restarted. It reads in the CSV file and sets a lot of other important variables that will be used later on.

The analysis begins at section 3; while each section has its own separate topic/focus, sections 3 and 4 may be needed to run the other sections later on. The sections can be categorized as such:




Table of Contents


1. Setup


2. CSV loading

3. Basic User Stats

4. Openings

5. Personal Opening Tree

6. Engine Evaluation

7. Time Analysis

8. Seasons

9. Win Predictor



1. Setup

Read here first!

Setup takes in a PGN file generated from https://lichess.org/api, and is meant to analyze the games of one user. So anyone using this should download their PGN by going to Games -> Export games of a user.

There are many parameters for download; it is important that you use the rated tag! Using casual games will not work with the current setup to build a CSV. Additionally, only standard games are allowed! No variants like bughouse, chess960 or antichess. The setup will error if it detects a non-standard game, and this can be avoided by setting the perfType to include any combination of blitz, bullet, classical or rapid.

The following tags should also be set to true to get the most out of the analysis: clocks, evals (make sure to analyze your games on Lichess!), opening.

TLDR; you may also just use the link provided below, after changing USERNAME to your LiChess username:

https://lichess.org/api/games/user/USERNAME?clocks=true&evals=true&opening=true&rated=true&perfType=classical,blitz,bullet,rapid

Input: PGN file

Output: Sorted CSV to create a data frame with the following columns:

1. "Year", "Month", "Day"
2. "Color"
3. "Result"
4. "Moves"
4. "Opponent"
5. "Opening", "Variation", "ECO", "ECO1", "ECO2"
6. "Time", "Increment", "Event"
7. "Rating", "OpponentRating", "RatingChange"
8. "WhiteMoves", "BlackMoves"
9. "WhiteRating", "BlackRating", "WhiteStockfish", "BlackStockfish"
10. "WhiteSeconds", "BlackSeconds", "WhiteMinutes", "BlackMinutes", "WhiteTotalSeconds", "BlackTotalSeconds", "WhiteTotalMinutes", "BlackTotalMinutes"
11. "Win", "Loss", "Draw"
12. The columns not needed but kept just in case: "Site", "UTCDate", "UTCTime", "Termination"

Or, an alternate view:

['Year', 'Month', 'Day', 'Color', 'Result', 'Opponent', 'Opening', 'Variation', 'ECO', 'ECO1', 'ECO2', 'Time', 'Increment', 'Event', 'Rating', 'OpponentRating', 'RatingChange', 'WhiteMoves', 'BlackMoves', 'WhiteRating', 'BlackRating', 'WhiteStockfish', 'BlackStockfish', 'WhiteSeconds', 'BlackSeconds', 'WhiteMinutes', 'BlackMinutes', 'WhiteTotalSeconds', 'BlackTotalSeconds', 'WhiteTotalMinutes', 'BlackTotalMinutes', 'Site', 'UTCDate', 'UTCTime', 'Termination']

Once you have done that, you will have a PGN file downloaded on your computer. Move that to the same folder as this ipynb file, and type in the name of the file without .PGN in the cell below.

Ex. if the file was named "Sicilian.pgn", then set the variable FILE_NAME to "Sicilian".

Also input your Lichess username. Capitalization matters.

Ex. if your username is "FriedLiver", then set the variable USERNAME to "FriedLiver".

To build the CSV from the given format, many adjustments were needed:

1. parse_pgn(): Transform the first 17/18 row entries into columns with regex
1. Entry 0, check rated; else error
2. Entries 1-16
3. Check that all are standard; else error
2. parse_notation(): 18th row, game notation
1. Game moves
2. Engine Rating
3. Move times (part I)
3. time_fn(): Handling move time increment
1. Time Control -> time and increment columns
2. Move times (part II)
4. format_df(): Formatting columns
1. "Player view" - Player Color, Opponent, Result, Elo, Rating Diff
2. Date/time related columns
3. Opening - splitting ECO/opening/variation
4. Reorder all columns

Common Errors:

Due to the nature of the PGN files downloaded from LiChess API, many errors are simply unavoidable without significantly more code during parsing, and so they must be dealt with through the PGN file. To deal with these errors, I suggest having a program like Notepad++. The chance of an error is roughly ~70% for games that have over 1000 games, based on 20 accounts tested.

The majority of errors fall within the first 2 categories.

  1. You played a player with a FIDE title before. The PGN contains titles of players, like [WhiteTitle "FM"]. There are too many titles as possibilities, including LM, NM, CM, FM, IM, GM, BOT, and all the arena and women's titles as well. Sometimes the PGN also includes [FEN ...] and [Setup ...]. These need to be removed manually in a program like Notepad++, using find and replace. In Notepad++, you can enable "regular expression" under search mode and enter both of the regex expressions in the cell below (one at a time) into the "find what" field, and leave "replace with" empty. Replace and they should all be gone.

  2. If setup is still failing, it's most likely going to be that a game doesn't track rating difference in the PGN, so [WhiteRatingDiff ...] and [BlackRatingDiff ...] are missing. A regex is provided below to find games where this occurs. Manually remove the game or fix it in Notepad++.

  3. There are events like "Italian Game Superblitz Arena" that have a different format on the PGN, and they all need to be removed manually as well if they error. Not all of these error, events like "Hourly/Weekly SuperBlitz Arena" are fine.

  4. The variant is not "Standard" even after filtering by events, and very common is "From Position". All the games with this need to be removed.

Run the cells below to generate a CSV. Drag the CSV into the folder with this ipynb file (if it isn't already), and proceed to later steps.


Important: once you have the CSV, you do not need to run Setup on any of these games again. If you have additional games to add in the future, you can download those separately by specifying a time to start from on the API: "since" or "until", which represents the unix epoch time in milliseconds: https://www.unixtimestamp.com/index.php. Or, you can just run all your games on this again and replace the new CSV with your old one. If you already had an existing CSV and you ran section 1 again, the new generated CSV file will replace the old one if their names are the same.


And if you error, refer to the error guide above and then run the cells below again.

Run Setup.

(Optional): view the table (and check for potential errors).

Download csv file (go to File->Open or the folder to see it, or through local files).



2. CSV loading

Must be run before any analysis after this can be looked at. Set FILENAME to the name of the csv without ".csv".

Remember to change USERNAME every time as well.

Ex. if the file was named "Sicilian.csv", then set the variable FILENAME to "Sicilian".


Optional: run below to view dataframe.



3. Basic User Stats


Covers:

3.1. A user's win/loss across all modes and for each specific game mode

3.2. Rating Difference vs Rating Change

3.3.1. User Rating by Date

3.3.2. User Rating Over Time

3.4. Performance against opponent rating


3.1 Win/Loss


Goal: See winrate overall and for each game mode (Blitz, etc.). Useful knowledge to have for later graphs.



3.2 Rating Difference vs Rating Change


Goal: This plot isn't intended to show how the rating changes based on the rating difference; rather, it shows how the player chooses their matchmaking. Do they tend to play only players rated much higher than them? Or lower? Provides justification for a low/high winrate from the cells above.



3.3.1 User Rating By Date


Goal: View user's rating over time.

Note: X axis is not linearly proportional over time! Each x-tick represents a day where the user played at least one game, so this is more like user rating over games played.

Potential error: the below graph may not output the x-axis (date) in the correct order. If so, then run the cell below it, although it is a worse version of this below graph, in my opinion. The main difference is the below graph shows rating change over units of games played, while the other one shows rating change over units of time.



3.3.2 User Rating Over Time


(Optional): If the cell above ran fine, then you don't really need to run this. This graphs based on time as a numeric value, so if you play many of your games in one short time period, then it'll appear as a big jump in rating on the graph (assuming there is a rating change). The x-axis also might not be printed properly, so if something like 20185 or 20180 shows up, interpret it as the middle of 2018 or the beginning of 2018.

Goal: Same as 3.3.1.



3.4 Performance against opponent rating


Goal: View user's winrate against a certain rating.

Note: The rating is represented as a range, since otherwise a winrate can't be calculated. There are two functions:

 perfELO_reg() : predicts winrate based on opponent rating.

 perf_ELO() : shows winrate based on opponent rating.

Reminder: From section 2, you can also use the following tables: bullet, blitz, rapid, classical, white, black. The first four represent games played only in that game mode, and the last two represent all games where you played as that side. This applies to all functions where you can provide an argument, but it is especially useful here because ratings are not equivalent across game modes!


Set how wide you want each rating group to be (the range).



4. Openings


Covers:

4.1. Openings played as White/Black

4.2. Opening Variations


4.1 Openings played as White/Black


Goal: Provides a list of the openings a user has played as White or Black, and sorts them in order by openings played most often (GP = Games Played). Displays the top 20 openings for both sides. There are many columns that the user can draw information from. What to look for:

  1. REL: Shows the average points per game (PPG) scored using this opening minus the average points per game (PPG) of the user over all their openings on that side. Essentially, it represents how good the user plays that opening relative to all their other ones.
  1. Rating Difference: The part that may invalidate REL. Some players choose to play different openings against different ratings, typically playing more "drawish" or closed openings against better players and more aggressive or open openings against lower rated players. If you constantly lose with an opening because you're only playing it against higher rated players, that doesn't mean you're playing the opening worse compared to your other openings.
  1. Rating Change (average per game) / Total Rating Gain: The most important columns in my opinion. It combines both REL and rating difference to show how successful your opening is in your ELO/rating. Your expected rating gain against any opponent and any rating should be 0, because the rating system factors in how much better/worse an opponent is.
  1. Moves: Do games played under this opening tend to be long or short? If you are bad with fast games, you might want to consider picking openings that have a shorter number of moves on average.
  1. W/L/D: Probably the first thing you'll pick up on, anyway.


4.2 Opening Variations played as White/Black


Goal: Any particular variation of an opening suddenly come to mind after that long opening list? Or perhaps you want to compare how you do using all the variations of the Sicilian?

Input: Type in which side you are playing from, and what opening you would like to see.

ex. Typing in "Black" and "Sicilian Defense" will show games where you played the Sicilian as black.



5. Personal Opening Tree


Something similar to what you'd find on openingtree.com or lichess.com/analysis, but obviously a downscaled version in nearly every aspect.

The personal opening tree allows you to type in moves and shows all next moves you or your opponents played with the frequency each occurred at and their winrates; it also shows the opponent's ratings and stockfish ratings behind every move.

However, something that I found out only after I implemented this was that the eval for the exact same positions are often different on the PGN file! For example, 1. e4 has an eval of +0.03 on some games, while that exact same first move has an eval of +0.24 on other games. Anyway, if you really wanted a proper engine score, you could just use openingtree or lichess analysis, since they're much easier to use and much more accurate.

Goal: This is more helpful for seeing moves you played in the past (or your opponents), and cross-checking what you play versus what your opponent plays. Also, looking at it from your opponent's perspective might tell you moves lower rated players like to play versus higher rated players in the same position.

Problem: Disregard the rating related columns in this table. The ratings are wildly inaccurate right now as I was forced to take the max of all values given for rating related variables. Switching to aggregating with the median may reduce the error quite significantly, but it will never guarantee the fact that the rating may still be far off given the input. As this isn't the highlight of the personal opening tree and there was no potential for it to be perfect, I've stalled on converting to a median, which proved to be harder than I had expected. Might come back to it after I finish the rest.

Note: The opening tree will show blue on your move and red on theirs.

Input: Check 2 cells down.





Personal Opening Tree


INPUT:

SIDE: Enter what side you played on with "White" or "Black".

MOVES: Enter the sequence of moves so far, ex. ["e4", "e5", "Nf3"]



6. Engine Evaluation


Everything to do with engine evaluations.


Covers:

6.0. Setup

6.1. Engine Evaluation against Opponent

6.2. Opening vs Engine Evaluation

6.3. Opening Variation vs Engine Evaluation

  6.3.1. Graphed against opening

  6.3.2. Graphed by variation only

6.4. Engine Evaluation on Piece Moved

6.5. Specified Eval + Move Number Statistics


Note: Reminder that the analysis in this section isn't based on all the user's games, but only the subset that have engine evaluations in them.



6.0 Setup


It's likely that many PGN files won't come with many games that have %eval (stockfish computer analysis in them), so filter the csv to only contain games that have stockfish evaluation in them. Run the cell below.




This next function is needed for sections 6.1 - 6.3.


6.1 Engine Evaluation against Opponent


Goal: Analyze patterns against a specific player. For example, does the user perform worse in the early game against this player?

Important: blue lines represent the user playing as White (positive ratings are winning), while red lines represent the user playing as Black (negative ratings are winning).

Input: Type in opponent name below.


6.2 Opening vs Engine Evaluation


Goal: Analyze the patterns for how a game plays out with a specific opening. Is there any point in the game where the user often holds a lead but loses it? What typically happens after the opening finishes?

Input: Type in which color the user plays as and what opening they play.



6.3  Opening Variation vs Engine Evaluation


Considers all games of a specific opening variation (more in-depth version of the graphs from above).

There are two functions below:

Function 1. Graph all rating graphs under an opening, but have the ones played under the specified variation in CYAN. Same runtime as the graphs above.

Goal: Compare the game flow for an opening variation versus all other variations in that opening.

Function 2. ONLY graph games with the given opening and variation.

Goal: Same as 6.2, but for an opening variation.

Important: the variation is included in the PGN file as the part after the ":". If no ":" appears for a game entry, then the variation is simply considered "Main Line".


Input: type in which color the user plays as and what opening/variation below, then run the cell after.




Function 1: graphs all games under the opening but the ones under the variation are colored in CYAN.




Function 2: graphs ONLY games under the variation in addition to opening.


6.4 Engine Evaluation on Piece Moved


Goal: What piece does the user blunder with the most? The table shows what pieces the user is good/bad with, and a "Total Gain" score that indicates the total amount of rating the user has gained on their opponents over all their piece moves. The graph shows where in the game they tend to blunder their pieces.

Note: It is better to have higher rating as White, and lower as Black. The reason it will be all negative for White and all positive for Black is because it is not possible to increase your evaluation score with a move (at best, you can only maintain the same engine evaluation). Of course, there will show instances where this happened because the data is not perfect.

(Optional) Note: On a more technical note, the "Total Gain" score is merely an estimate, not the real gain you get across all moves. The true total gain could be calculated by multiplying the user_count avg_user_rating - opponent_count avg_opponent_rating, but if the user tends to play much more bishop moves, then the total gain will probably be worse for the user (why? see above note). That's not because the user is worse with the piece but because they play it more often. Sure, the average rating already reflects this, so why does it matter? My intentions behind showing the "Total Gain" was to give users an idea of significant their average loss was compared to their opponents by factoring in how often the piece is played from both sides. For example, making mistakes on castling should result in low total gain because it can only be played once per game. I did not want to factor in how much a user chooses to move a piece vs how often their opponent does as well.

Input: Type in which color the user plays as and what opening they play.



6.5 Specified Eval + Move Number Statistics


Given a certain eval rating and a move number, what is the winrate for the user?

Goal: There's not much to see here, but maybe you might find that your winrate is horrificly low under a certain engine evaluation, like -2 (indicating that you give up too easily). Or that you choke through leads a lot. Besides that, it's just a fun tool to try out.

Important: In addition to outputting winrate, you can also optionally output a dataframe and a list that shows the index of the games where the user converted a win or loss. For example, if you wanted to see games where you came back from a -5 engine evaluation, you could use the list of indices / dataframe and match them with your CSV. To match the indices with the CSV, you will have to add 2 to each index (the actual CSV file's first entry is at row 2, while it is 0 in our dataframe).

Inputs: A lot! Check the cell below for instructions.


After inputs are ready, run the cell below.



7. Time Analysis


Everything to do with clock times.


Covers:

7.0. Setup

7.1. Move # vs Time Spent

7.2. Move # vs Time Remaining

7.3. Engine Evaluation vs Time Spent

7.4. Move Rating vs Time Spent

7.5. Piece Moved vs Time Spent


(Optional) Note: I was going to add something time control + increment related as well but the reason I didn't was because I don't think showing time+increment vs winrate shows anything significant. Having a lower winrate in 5 minute games versus 15 minute games is more of a Blitz vs Classical issue, and the equivalent rating in Blitz and Classical are not the same, ex. a 1500 player in Blitz is better than a 1500 player in Classical.



7.0 Setup


Like with before, many games won't have time evals either, so filter by games that have time evals.




The general function needed for all the time graphs below. At the bottom of the cell below, specify what mode or time control + increment you want to look at for time analysis. If you want to include all games for any of them, just leave the variable as a "", ex. time = "". If you want a 10+5 control, then set time=10 and increment=5.


Note: the PGN often didn't provide moves or ratings past move 99, so they any 100+ move/ratings likely won't appear in the graphs later.


7.1 Move # vs Time Spent


Compares the average time per move for a user and all opponents they played against.

Goal: See where in the game the user typically spends the most time thinking, and how they compare overall against opponents.



7.2 Move # vs Time Remaining


Compares the average time left for a user and all opponents they played against.

Goal: See how a user chooses to budget their time in a game.



7.3 Engine Evaluation vs Time Spent


Compares the current engine evaluation and how much time is spent on that move.

Goal: There's not much "good" or "bad" to see from this graph, but a difference between the user and opponent may say something about the user's habits when they are up or down on in the game.



7.4 Move Rating vs Time Spent


Compares the move's engine rating and how much time is spent on that move.

Goal: The focus on this graph is in the bad moves, or moves with negative ratings. Does the user typically spend less time thinking before making those moves? Or do they think a long time and still make the wrong move? Blunders are hard to make with a lot of thought, so if the user's curve is under the opponent's curve then they aren't spending enough time on moves (resulting in blunders).



7.5 Piece Moved vs Time Spent


Produces a graph for each piece and graphs move number vs the amount of time is spent in moving that piece.

Goal: There are many graphs here and the lines aren't very clear, but any extremely noticeable patterns indicate that the user has a particular habit with regards to moving a certain piece at a certain point in the game.




8. Seasons


Covers:

8.1. Seasons

8.2. Season Graphs


Feel free to skip this section, it's a very specific system that tracks data more than it really shows anything. Or you can run it if you want.

How I play rated on Lichess: every "season" consists of N=50 rated games, where I play in a specific rating range of range 150 (max - min) for Classical/Rapid only. After 50 games, the PPG (points per game) I get determines whether I keep the range the same, or promote/demote the range by +/-(50 * n) ELO points. For example, I might be playing on 1550-1700 in one season, and after promoting on a 29-19-2 season, I would be playing in the 1600-1750 range the next season.

Goal: Of course, since nobody else probably plays like this, the information above won't be very useful. Instead, what you can get out of this section is your performance for every N games (and especially, the most recent N games). It may also show how a user's opening repertoire has changed over time.


Note: used recursion to have the most recent season print first.

Inputs: N represents how long a season is (or every N games to show), and startELO and endELO represent the starting range for the first season.



8.1 Seasons



8.2 Season Graphs


The following cell plots the season in the x-axis, the average opponent ELO in the y-axis, and the score (wins + 0.5 * draws) in the z-axis. These were defined as x, y and z in the previous function and returned as the output of season_fn.


Non-interactive version:

Interactive version:

2D version:



9. Win Predictor using PCA + Multinomial Logistic Regression


Covers:

9.0. PCA + Model

9.1. Model 1: Predictor using model_df

9.2. Model 2: Predictor without game knowledge

9.3. Model 3: Predictor with opening knowledge




9.0 PCA + Model


PCA

Output: Plotted the impact of each variable and a scree plot.

"PCA really shines on data where you have reason to believe that the data is relatively low in rank."

Results may differ based on what chess dataset you use, but it seems that in virtually all cases, all the variables used for PCA seem to be indepedent, or in other words, the data is already high in rank and cannot be reduced further down.

Was surprised to see that "Color" and "Opponent Rating" had relatively low values. "Color" may have been somewhat affected by the opening played, and the ratings may have been covered in part by "Event", as ratings do differ quite drastically across different game modes.

Note: If you don't have model_df or don't want to use it, comment out the last two lines of the cell below (with #) and skip section 9.1.


Model

Output: a logistic regression model and a linear regression model.

The logistic regression model has to be able to output 3 different results, from Win, Draw, or Loss, so I used a multinomial logistic regression model for this. Therefore, the random guess has now a 33% chance of being correct instead of 50% correct.

The linear regression model outputs the expected PPG (points per game).

One hot encoding was used to transform categorical variables into numerical ones, and the train-test split used was 80-20, although the train and test accuracy were very similar for any values used from a 65-35 split to a 90-10 split.




9.1 Model 1 : Predictor using model_df


This model uses model_df as training dataset for the model. model_df contains over 49,000 games, with at least 5000 in each of the four game modes. It also spans a rating range of around 1000 to 3000, which I tried to balance evenly for each of the four modes.

Inputs: The user enters the side a user plays on, the user's rating, the opponent's rating and the game type.



9.2 Model 2 : Predictor without game knowledge


This model uses the user's own games to make a prediction.

Inputs: The user enters the side a user plays on, the time control, the opponent's rating and the game type.



9.3 Model 3 : Predictor with opening knowledge


This model uses the user's own games to make a prediction.

Inputs: The user enters the side a user plays on, the time control, the opponent's rating, the game type, and also the ECO letter code.